ROW-CONTINUOUS FINITE MARKOV CHAINS : STRUCTURE AND ALGORITHMS

Continuous Time Markov Chains

Discrete-time Markov chains are useful in simulation, since updating algorithms are easier to construct in discrete steps. They can also be useful as crude models of physical, biological, and social processes. However, in the physical and biological worlds time runs continuously, and so discrete-time mathematical models are not always appropriate. This is especially true in population biology –...

Continuous-time Markov Chains

Model-Checking Algorithms for Continuous-Time Markov Chains

Continuous-time Markov chains (CTMCs) have been widely used to determine system performance and dependability characteristics. Their analysis most often concerns the computation of steady-state and transient-state probabilities. This paper introduces a branching temporal logic for expressing real-time probabilistic properties on CTMCs and presents approximate model checking algorithms for this ...

Finite Markov Chains

This paper provides basic information and theorems about finite Markov chains. The inspiration for this paper came from Professor Laci Babai’s Discrete Mathematics lecture notes from the REU program of 2003. After reading through his chapter on Markov chains, I decided to proceed by answering as many exercises from the notes as possible. Below is what I have finished.

Singularly Perturbed Finite Markov Chains with General Ergodic Structure

We analyse singularly perturbed Markov chains. Most previous research has been done under the assumption that the perturbed Markov chain is either ergodic or unichain. In this paper we do not impose any restrictions on the ergodic structure of the perturbed chain. The present approach is based on the inversion of analytic matrix-valued functions.